Some kinds of deep neural network have layers of very different forms and learning rules in addition to simple neurons with back propagation. For example, the early layers might use a Restricted Boltzmann Machine, a convolutional neural network for images; or a transformer Models for text. Similarly memory layers may be used as inner layers. The term is not usually used if the neural network architeture simply varies in the number of nodes at each layer, for example including one or more pinch points. Also where the layers in system include other forms of AI, for example neural network that feeds its output into a rule-based system, this would be regarded as a hybrid architecture or neurosymbolic AI, not just hybrid layers.
Used on Chap. 6: page 125